from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.run()

KNeighborsClassifier_brute_force

onnx (1.9.0) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.648	0.127	0.0	0.019	-1	1	0.639	2.240	0.042	8.324	0.166	See	See
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.648	0.127	0.0	0.019	-1	1	0.639	0.028	0.004	666.736	89.491	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.358	0.013	0.0	0.358	-1	1	1.000	2.240	0.042	0.160	0.006	See	See
3	KNeighborsClassifier_brute_force	predict	100000	1	100	0.358	0.013	0.0	0.358	-1	1	1.000	0.028	0.004	12.783	1.773	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.431	0.181	0.0	0.018	1	1	0.639	1.576	0.009	11.693	0.132	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.431	0.181	0.0	0.018	1	1	0.639	0.025	0.001	722.838	41.965	See	See
6	KNeighborsClassifier_brute_force	predict	100000	1	100	0.366	0.010	0.0	0.366	1	1	1.000	1.576	0.009	0.233	0.007	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1	100	0.366	0.010	0.0	0.366	1	1	1.000	0.025	0.001	14.372	0.914	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.977	0.335	0.0	0.019	1	100	0.851	2.622	0.027	7.238	0.148	See	See
9	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.977	0.335	0.0	0.019	1	100	0.851	0.028	0.002	678.218	40.318	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1	100	0.363	0.007	0.0	0.363	1	100	1.000	2.622	0.027	0.138	0.003	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.363	0.007	0.0	0.363	1	100	1.000	0.028	0.002	12.967	0.780	See	See
12	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.673	0.011	0.0	0.019	-1	100	0.851	3.234	0.128	5.774	0.229	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.673	0.011	0.0	0.019	-1	100	0.851	0.027	0.002	704.090	63.234	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.330	0.008	0.0	0.330	-1	100	1.000	3.234	0.128	0.102	0.005	See	See
15	KNeighborsClassifier_brute_force	predict	100000	1	100	0.330	0.008	0.0	0.330	-1	100	1.000	0.027	0.002	12.435	1.153	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.038	0.340	0.0	0.019	-1	5	0.720	3.274	0.062	5.815	0.152	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.038	0.340	0.0	0.019	-1	5	0.720	0.031	0.003	617.592	67.720	See	See
18	KNeighborsClassifier_brute_force	predict	100000	1	100	0.362	0.008	0.0	0.362	-1	5	1.000	3.274	0.062	0.110	0.003	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1	100	0.362	0.008	0.0	0.362	-1	5	1.000	0.031	0.003	11.731	1.298	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.049	0.045	0.0	0.019	1	5	0.720	2.598	0.054	7.333	0.154	See	See
21	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.049	0.045	0.0	0.019	1	5	0.720	0.026	0.001	732.241	41.788	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1	100	0.366	0.010	0.0	0.366	1	5	1.000	2.598	0.054	0.141	0.005	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	100	0.366	0.010	0.0	0.366	1	5	1.000	0.026	0.001	14.078	0.884	See	See
24	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.367	0.057	0.0	0.004	-1	1	0.831	1.810	0.044	2.413	0.066	See	See
25	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.367	0.057	0.0	0.004	-1	1	0.831	0.010	0.004	436.952	158.955	See	See
26	KNeighborsClassifier_brute_force	predict	100000	1	2	0.276	0.013	0.0	0.276	-1	1	1.000	1.810	0.044	0.152	0.008	See	See
27	KNeighborsClassifier_brute_force	predict	100000	1	2	0.276	0.013	0.0	0.276	-1	1	1.000	0.010	0.004	27.590	10.108	See	See
28	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.290	0.122	0.0	0.004	1	1	0.831	1.276	0.012	3.362	0.101	See	See
29	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.290	0.122	0.0	0.004	1	1	0.831	0.002	0.000	1913.633	168.278	See	See
30	KNeighborsClassifier_brute_force	predict	100000	1	2	0.284	0.008	0.0	0.284	1	1	1.000	1.276	0.012	0.223	0.006	See	See
31	KNeighborsClassifier_brute_force	predict	100000	1	2	0.284	0.008	0.0	0.284	1	1	1.000	0.002	0.000	126.655	11.082	See	See
32	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.383	0.145	0.0	0.004	1	100	0.892	2.137	0.034	2.051	0.075	See	See
33	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.383	0.145	0.0	0.004	1	100	0.892	0.003	0.000	1373.097	141.126	See	See
34	KNeighborsClassifier_brute_force	predict	100000	1	2	0.272	0.008	0.0	0.272	1	100	1.000	2.137	0.034	0.127	0.004	See	See
35	KNeighborsClassifier_brute_force	predict	100000	1	2	0.272	0.008	0.0	0.272	1	100	1.000	0.003	0.000	85.183	8.693	See	See
36	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.306	0.117	0.0	0.004	-1	100	0.892	2.753	0.052	1.564	0.052	See	See
37	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.306	0.117	0.0	0.004	-1	100	0.892	0.010	0.005	430.975	216.161	See	See
38	KNeighborsClassifier_brute_force	predict	100000	1	2	0.286	0.009	0.0	0.286	-1	100	1.000	2.753	0.052	0.104	0.004	See	See
39	KNeighborsClassifier_brute_force	predict	100000	1	2	0.286	0.009	0.0	0.286	-1	100	1.000	0.010	0.005	28.646	14.373	See	See
40	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.214	0.073	0.0	0.004	-1	5	0.875	2.722	0.042	1.548	0.036	See	See
41	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.214	0.073	0.0	0.004	-1	5	0.875	0.010	0.002	405.126	91.178	See	See
42	KNeighborsClassifier_brute_force	predict	100000	1	2	0.267	0.007	0.0	0.267	-1	5	1.000	2.722	0.042	0.098	0.003	See	See
43	KNeighborsClassifier_brute_force	predict	100000	1	2	0.267	0.007	0.0	0.267	-1	5	1.000	0.010	0.002	25.690	5.807	See	See
44	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.260	0.076	0.0	0.004	1	5	0.875	2.183	0.035	1.952	0.047	See	See
45	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.260	0.076	0.0	0.004	1	5	0.875	0.003	0.000	1351.599	91.935	See	See
46	KNeighborsClassifier_brute_force	predict	100000	1	2	0.284	0.008	0.0	0.284	1	5	1.000	2.183	0.035	0.130	0.004	See	See
47	KNeighborsClassifier_brute_force	predict	100000	1	2	0.284	0.008	0.0	0.284	1	5	1.000	0.003	0.000	90.218	6.482	See	See

KNeighborsClassifier_kd_tree

onnx (1.9.0) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	136.319	0.000	0.0	0.136	-1	100	0.973	3.441	0.047	39.611	0.540	See	See
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	136.319	0.000	0.0	0.136	-1	100	0.973	0.012	0.001	11827.271	1251.306	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.046	0.125	0.0	3.046	-1	100	1.000	3.441	0.047	0.885	0.038	See	See
3	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.046	0.125	0.0	3.046	-1	100	1.000	0.012	0.001	264.307	29.994	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	130.302	0.000	0.0	0.130	1	5	0.972	1.702	0.013	76.566	0.595	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	130.302	0.000	0.0	0.130	1	5	0.972	0.004	0.001	34635.696	11544.383	See	See
6	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.210	0.149	0.0	3.210	1	5	1.000	1.702	0.013	1.886	0.089	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.210	0.149	0.0	3.210	1	5	1.000	0.004	0.001	853.278	287.155	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	134.010	0.000	0.0	0.134	-1	5	0.972	1.014	0.025	132.150	3.310	See	See
9	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	134.010	0.000	0.0	0.134	-1	5	0.972	0.008	0.002	17451.287	3679.085	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.130	0.118	0.0	3.130	-1	5	1.000	1.014	0.025	3.086	0.139	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.130	0.118	0.0	3.130	-1	5	1.000	0.008	0.002	407.578	87.280	See	See
12	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	138.616	0.000	0.0	0.139	1	100	0.973	5.796	0.065	23.915	0.268	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	138.616	0.000	0.0	0.139	1	100	0.973	0.009	0.001	15881.616	2662.449	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.143	0.199	0.0	3.143	1	100	1.000	5.796	0.065	0.542	0.035	See	See
15	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.143	0.199	0.0	3.143	1	100	1.000	0.009	0.001	360.134	64.518	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	138.737	0.000	0.0	0.139	-1	1	0.962	0.522	0.009	265.850	4.753	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	138.737	0.000	0.0	0.139	-1	1	0.962	0.006	0.002	24980.937	7353.665	See	See
18	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.120	0.101	0.0	3.120	-1	1	1.000	0.522	0.009	5.979	0.222	See	See
19	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.120	0.101	0.0	3.120	-1	1	1.000	0.006	0.002	561.837	166.391	See	See
20	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	140.717	0.000	0.0	0.141	1	1	0.962	0.902	0.015	155.949	2.517	See	See
21	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	140.717	0.000	0.0	0.141	1	1	0.962	0.002	0.000	60818.772	10950.878	See	See
22	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.177	0.070	0.0	3.177	1	1	1.000	0.902	0.015	3.521	0.096	See	See
23	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.177	0.070	0.0	3.177	1	1	1.000	0.002	0.000	1373.120	249.062	See	See
24	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.076	0.014	0.0	0.000	-1	100	0.916	0.049	0.004	1.544	0.310	See	See
25	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.076	0.014	0.0	0.000	-1	100	0.916	0.004	0.001	21.211	5.536	See	See
26	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	100	1.000	0.049	0.004	0.131	0.015	See	See
27	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	100	1.000	0.004	0.001	1.799	0.362	See	See
28	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.048	0.001	0.0	0.000	1	5	0.909	0.032	0.001	1.478	0.044	See	See
29	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.048	0.001	0.0	0.000	1	5	0.909	0.001	0.000	54.832	12.343	See	See
30	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	5	1.000	0.032	0.001	0.195	0.010	See	See
31	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	5	1.000	0.001	0.000	7.243	1.660	See	See
32	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.048	0.001	0.0	0.000	-1	5	0.909	0.037	0.002	1.279	0.076	See	See
33	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.048	0.001	0.0	0.000	-1	5	0.909	0.003	0.001	14.053	3.601	See	See
34	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	5	1.000	0.037	0.002	0.171	0.012	See	See
35	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	5	1.000	0.003	0.001	1.882	0.487	See	See
36	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.071	0.002	0.0	0.000	1	100	0.916	0.045	0.001	1.579	0.069	See	See
37	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.071	0.002	0.0	0.000	1	100	0.916	0.001	0.000	80.780	14.938	See	See
38	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	100	1.000	0.045	0.001	0.141	0.008	See	See
39	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	100	1.000	0.001	0.000	7.193	1.352	See	See
40	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.046	0.001	0.0	0.000	-1	1	0.890	0.032	0.003	1.408	0.144	See	See
41	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.046	0.001	0.0	0.000	-1	1	0.890	0.003	0.000	16.005	1.790	See	See
42	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.007	0.001	0.0	0.007	-1	1	1.000	0.032	0.003	0.209	0.027	See	See
43	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.007	0.001	0.0	0.007	-1	1	1.000	0.003	0.000	2.371	0.326	See	See
44	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.045	0.002	0.0	0.000	1	1	0.890	0.029	0.001	1.554	0.110	See	See
45	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.045	0.002	0.0	0.000	1	1	0.890	0.001	0.000	52.110	20.935	See	See
46	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	1	1.000	0.029	0.001	0.208	0.013	See	See
47	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	1	1.000	0.001	0.000	6.986	2.798	See	See

HistGradientBoostingClassifier_best

onnx (1.9.0) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	n_iter	mean_duration	std_duration	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.545	0.023	0.001	0.001	0.815	0.184	0.005	2.961	0.147	See	See
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.545	0.023	0.001	0.001	0.815	0.022	0.001	24.473	1.792	See	See
2	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.421	0.017	0.000	0.421	1.000	0.184	0.005	2.288	0.108	See	See
3	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.421	0.017	0.000	0.421	1.000	0.022	0.001	18.908	1.355	See	See